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ABSTRACT 

A study evaluated reaching achievement for three 
groups of first-grade students: (1) those receiving phonics 

instruction utilizing the "Open Court" phonics materials integrated 
with the District’s language arts curriculum; (2) those receiving 
phonics instruction in the context of guided and shared reading 
activities ("Phonics in Context"); and (3) a control group receiving 
no special program beyond normal instruction. Reading pretest results 
and a measure of socio-economic status revealed remarkable similarity 
among the groups. In regard to recent staff development and 
experience with first-grade curriculum, there was little difference 
between the teachers involved in the study. End of year posttest 
reading achievement showed no significant differences among the three 
groups on the Test of Early Reading Ability-2nd Edition, and the same 
was true for the Running Record assessment that measured level of 
oral reading proficiency. The Stanford Achievement Test showed no 
significant differences between the Open Court and Phonics in Context 
groups, but did show significant differences between Open Court and 
the Control group on all three subtests. The Phonics in Context group 
also outperformed the control group on the Stanford Word Study Skills 
subtest as well as on the Stanford Total Reading score. Results of a 
parent survey showed high levels of enthusiasm and parent 
satisfaction for each of the reading programs. Teacher interviews 
revealed high levels of enthusiasm for both treatment programs, 
(Contains six tables and three figures of data.) (Author/SR) 
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ABSTRACT 



The Language Arts First Grade Field-test evaluation focused on reading 
achievement for three groups of students: (1) Open Co urt Integrated Model: those 
receiving phonics instruction utilizing the Open Court phonics materials integrated 
with the District's language arts curriculum, (2) tf -se receiving phonics instruction 
in the context of guided and shaied reading activities (Phonics in Context 
Intem-ated Model) , and (3) a Control Ef roup of students at school receiving no special 
program heyond that which occurs through the school's on-going staff 
development. 

Reading pretest results obtained at the beginning of first grade, and a 
measure of socio-economic status (SES) revealed remarkable similarity between 
the three groups, thus making a direct comparison of their post-test reading scores 
feasible. In addition to SES, and pre-test reading readiness, information was 
collected regarding the background of the teachers. In regard to recent staff 
development and experience with the first grade curriculum, there appeared to be 
little difference between the teachers involved in the study. 

End of year post-test reading achievement showed no significant differences 
between the three groups on the Test of Early Reading Ability-2nd 
Edition(TERA2). "nie Running Record assessment that measured level of oral 
reading proficiency showed the same pattern of small but statistically insignificant 
differences between the groups. The third measure, the Stanford Achievement 
Test, showed no statistically significant differences between the Open Court and 
Phonics in Context groups but did result in significant differences between Open 
Court and the Control group on all three Stanford reading sub-tests: Reading 
Comprehension, Word Reading, and Word Study Skills. The Phonics in Context 
group also outperformed the Control group on the Stanford Word Study Skills sub- 
test as well as on the Stanford Total Reading score. 

Results of a parent survey showed high levels of enthusiasm for each of the 
reading programs with no significant group differences reported in parent 
satisfaction with student progress in reading. Teacher interviews revealed high 
levels of enthusiasm for both treatment programs although the on-going delivery 
of in-service to the Phonics- in-Context group throughout the year made it unlikely 
that the full effects of the program could be determined from end of year results. 



Language Arts Field Test 



OVERVIEW 

The purpose of the First Grade Language Arts Field Test was to provide information which 
will lead to improved reading instruction and achievement for all RUSD students. Since 
the new state English Language Arts instructional materials adoption will occur in 1996, 
one purpose of the field test was to provide the district with an informed basis for malting 
decisions regarding instmctional materials. Equally important was the opportunity to 
improve our understanding of effective reading instruction and English Language Arts 
implementation. 

Three sets of first grade, English-only classrooms, participated in the field test representing 
two treatment groups (Open Court and Phonics in Context) and a control group. Selection 
of classes for the three groups was based on teacher and school readiness and teacher 
interest in participating. An attempt was made to have representation across the district 
within each set and, to the extent possible, to involve schools with similar percents of 
childi en receiving a free or reduced price lunch. Principals were asked to i ecommend 
experienced teachers who could be expected to fully implement the respective field test 
programs. The final selection resxolted in five Open Court, seven Phonics in Context, and 
four Control group classrooms participating. Table 1 shows the program design. 

TABLE 1 



Program Design 



Proeri’am Parts 


Open Court 


Phonics In Context 


Control 




Integrated Model 


Integrated Model 






Core Literature 


Core Literature 


Core Literature 


Materials 


Spelling Through 
Phonics 


Spelling Through 
Phonics 


Spelling Through 
Phonics 




Published Phonics 
Kit 


District Phonics 
Materials/Program 




Advanced 

Training 


Sustained Ti'aining 
Program 


Sustained Training 
Program 




Designated 

Support 

Strategies 


Reading Materials 
Sent Home Daily 


Reading Materials 
Sent Home Daily 





PROGRAM DESCRIPTIONS 

Open Court Phonics — This field test program focused on the use of the phonics materials 
and strategies developed by Open Court Publishing and the integration of those materials 
with our district’s adopted literature program. Foil ow-up observations and coaching were 
also included. 
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The integrated Open Court Phonics and literature model required the purchase of the 
Open Court Phonics Kit; Alphabet-Sound cards, Teacher's Resource Binder, and the 
Storybook Kit. 

The integrated Open Court Phonics and literature model was already being implemented 
in the a nearby school district. Specifically, the model includes systematic instruction in 
phonics and decoding skills throughout first grade using all of the materials listed above. 
Implementation for Open Court consisted of a scheduled set of daily activities which 
included 60 minutes of phonics supplemented by other literacy activities. Students took 
home reading material each day. Teachers were to integrate phonics instruction with both 
the Houghton-Mifflin literature series and the Wright and Rigby reading books to promote 
independent student read'.ng. This model is in alignment with the language arts 
instructional focus of our K-6 schools and K-3 Reading and Writing classroom assessment 
standards. These standards focus on instruction in phonics, spelling, the identification of 
resources v/hich will support all students being able to select books at a comfortable 
reading level, and the systematic monitoring of student progress. 

Staff development included two full-days of training in late August followed by three 
aRernoon follow-up sessions in October, February, and April. Least one on-site visitation 
to the classrooms. The cost of this staff development was $500 per day. 

Phonicsin Context — The Phonics in Context ^oup included teachers who were completing 
the second year of the RUSD K-3 Literacy Training Program. Phonics in Context required 
eight sets of Rigby or Wright group Big Books and Little Books at a cost of $400 - $500 per 
classroom. Houghton-Mifflin literature and Wright andRigby readingbooks were used to 
promote independent reading in both sets of classrooms. 

The Phonics in Context model was developed by district teachers. Staff Development 
Specialists, Instructional Services Specialists, and mentor teachers. It matches the 
recommendations in the California Department of Education document entitled “Ready, 
Set, Read” and focuses on developing a reading program which allows for extension of 
student learning, opportunities for students to interact with difficult text, andopportiimties 
for students to practice on “independent” reading level texts. The major instructional 
activities for Phonics in Context were Shared Reading and Guided Reading. Letter-sound 
relationships (phonics) were emphasized using these strategies. Students took home 
independent reading materials each day. The 1994-95 school year was the first time this 
training program was implemented. 

District Staff Development Specialists and mentor teachers provided three full-days of 
staffdevelopment from September 30 to October 28 and another full-day in both March and 
April. Half-day inservices were provided in December, January, and February. These 
demonstration lessons utilized shared and guided reading strategies and emphasized skiU 
development. Teachers were further supported by classroom visitations and coaclung as 
well as on-site demonstration lessons provided by District Staff Development Specialists. 

Control Group — The teachers in the control group did not receive any special training. 
They continued their implementation of the RUSD first grade English Language Arts 
course of study and only participated in school site staff development. Students in the 
control group were assessed, however, in the same manner as for the other two groups. As 
with the other two groups, an effort was made to match the classes demographicaUy. 
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DATA COLLECTION 



Data collection extended from September 1994 through June, 1995. Reading achievement 
outcome measures were varied and included a widely used nationally normed reading test, 
an individually administered test of early reading ability, and a teacher scored assessment 
of oral reading proficiency used in the district’s Reading Recovery program. In addition to 
reading achievement measures, parents were surveyed to ascertain their perceptions of 
their child’s progress and to provide information about their child’s reading beha^dor at 
home. 

To gather information about the programs from those involved, teachers from the Open 
Court and Phonics in Context programs were interviewed along with the St^Development 
Specialists assigned to support the implementations. Finally, to determine background 
characteristics of the teachers and characteristics of their classrooms, teachers were asked 
to complete a questionnaire administered in the fall. 

A short description of each instrument and the role it played in the evaluation follows: 

Data from Students : 

(1) Test of Early Reading Ability-2nd Edition (TERA-2) Pretest/Posttest 

The TERA-2 (PRO-ED, 1991) was administered as a pretest in the fall and also as aposttest 
in the spring. This individually administered test of early reading skills is highly reliable 
and provides information on progress made throughout the year. It could also be used to 
adjust the outcome measures for student differences in reading readiness that existed at 
the beginning of the program, if necessary. 

(2) Reading Recovery Running Record Pretest/Posttest 

Taken from the Reading Recovery (Clay, 1979) assessment battery, the Running Record 
was individually administered by teachers in a natural setting during both the first and 
third trimesters using high interest transitional books. The student’s highest level of oral 
reading ability at 90 percent or higher accuracy was identified using books that were 
leveled from 1 to 9 by one of the district’s Reading Recovery Specialists. 

(3) Stanford Achievement Reading Subtests (Word Reading, 

Word Study Skills, and Reading Comprehension) Posttest 

The Stanford Primary I Reading subtests (The Psychological Corporation, 1989) are 
nationally normed and group administered at the end of first grade only. The Word 
Reading subtest measures vocabulary while the Word Study Skills subtest focuses directly 
on phonics (decoding) skills. Reading Comprehension, of course, measures the ability of 
students to read short sentences and passages and answer questions about what they have 
read. 
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Data from Teachers : 

(1) Teacher Background Questionnaire 



The Teacher Background Questionnaire sought to provide information about teacher staff 
development training, years of teaching experience at the early grades (especially first 
grade), and characteristics of the teachers’ classrooms. The primary purpose was to 
ascertain if there were background differences that could be expected to affect reading 
achievement independent of the specific program implemented. 

(2) Teacher Interviews (Open Court and Phonics in Context only) 

The Open Court teachers were interviewed in a group setting in April to obtain feedback 
about their perceptions of the program. Likewise, the Phonics in Context teachers were 
also interviewed. 

Data from Parents : 

(1; Parent Telephone Survey 

Parents of students ir. all three groups were surveyed by phone and were requested to 
answer several questions dealing with the perception of their child's reading progress as 
well as questions relating to how much exposure to reading their child experienced at home. 

(2) Parent Occupation as a measure of Socio-Economic Status (SES) 

Parent occupational levels were coded using a five-point scale based on information 
provided by parents on the student registration record. The coding of parent occupational 
levels was completed by the teachers who administered the TERA-2 to students diuingthe 
spring. The categories were the same as those used by the California Assessment Program 
to gauge parent socio-economi c status — a variable consistently found to be highly correlated 
with student achievement and one, therefore, that needs to be considered when looking at 
reading outcomes. 

Other : 

(1) Interviews with Staff Development Facilitators/Trainers 

The two, RUSD Staff Development Specialists, assigned to the program were interviewed 
to obtain their perceptions of both the Open Court and Phonics in Context program 
implementations, respectively. 

(2) Phone Interview with Open Court Trainer 

The Open Court trainer, was interviewed by phone during the same interval as the Staff 
Development facilitator interviews. 
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DATA ANALYSIS PROCEDURES 



Comparisons of the reading scores for the three groups were made using analysis of 
variance. Both the pretest TERA.-2 data and the parent occupation information were 
analyzed to determine if covariate adjustments in end of year outcomes would be 
necessary. These two variables were expected to be substantially related to reading 
outcomes at the end of first grade and, therefore, need to be considered. The three reading 
measures con^is^ing of the Stanford Total Reading score, the TERA-2 score, and Running 
Record level, were taken as a composite in the initial multivariate analysis to define 
“reading achievement.” Follow-up analyses of the individual reading achievement 
measures were then conducted, as needed, to investigate which of the measures showed 
differences among the field test groups. 

Teacher profiles were developed for each of the three groups from the Teacher Questionnaire 
to provide information relating to possible differences in teacher instructional skills, past 
st^ development training, and classroom characteristics that could be expected to affect 
program outcomes. 

Parent telephone survey results were reported for each of the three groups by directly 
comparing the percents of parents responding to the various response options on the six 
questions asked. The average scores on the scales were then compared for the three groups 
using analysis of variance. 

Teacher and staff observations were summarized through narratives constructed from 
interview notes. 

EVALUATION RESULTS 

Description of the Sample 



The composition of the field test groups (Table 2 - Appendix A) was similar except for the 
number of Chapter 1 students where the Phonics in Context group had a larger number, 
and the number of Reading Recovery students where the Control group had a smaller 
number. Both Reading Recovery and Chapter 1 students are identified "high risk" 
students in reading and receive supplementary instruction in a small group (Chapter 1) 
or individual basis (Reading Recovery). 

The Parent Occupation Scale used in the study was the same as the indicator used by 
previous State assessments. The categories are as follows: 

1 - Unknown 

2 - Unskilled Employees 

3 - Skilled and Semi-skilled Employees 

4 - Technical, Sales, and Administrative Support Workers 

5 - Managers and Professionals 

Parent occupation, as is the case in most studies of this t>i)e, is substantially related to 
achievement outcomes. Student performance at the end of first grade is also highly related 
to student test scores at the beginning of first grade. As a consequence, it is necessary to 
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account for any group differences on these two variables when reporting performance 
levels on the reading tests. The correlations between the pre-test TERA-2 scores and the 
posttest measures as well as the correlation between SES (parent occupation) and the 
posttest measures are shown in Table 3 - Appendix A. These correlations have a possible 
range from 0 (no relationship) to 1.00 (perfect relationship). 

Table 4 - Appendix A shows the performance of the three field test groups on the Pretest 
TERA-2 and the measure of SES (parent occupation). It is clear that the groups were 
remarkably equivalent on both these measures, especially the Open Court and Phonics in 
Context groups. Tests of statistical significance of the cbfferences between ^oups were 
found to be non- significant. Consequently, there appeared to be no need to adjust posttest 
outcomes for these two measures using analysis of covariance and direct comparison of the 
three groups on the three reading measures was done in the interest of simplicity. 

Analysis of Results 

As described earlier, three instruments v;ere used to measure reading achievement in the 
field test: Test of Early Reading Ability (TERA-2), the Running Record, and the Stanford 
Achievement Test. The initial analysis tested the significance of the differences in reading 
achievement for the groups with reading defined as a composite of all three reading 
measures. The results of this multivariate analysis of variance showed that reading 
achievement differed significantly among the three groups defined in the field test. 
Consequently, follow-up univariate analyses were run to investigate (a) which groups 
differed significantly from one another, and (b) on which of the reading measures they 
differed (Table 5). 

Table 5 shows the results including those of the Stanford subtests. Because of the need to 
consider the pretest reading performance and SES, the analyses included only cases 
having a valid TERA-2 pretest score and a known parent occupation level. With the 
exception of the Running Record which is on a 1-9 sc'^’e, all results are in terms of normal 
cur\'e eqioivalent (NCE) scores which range from 1-99 with 50 indicating average 
performance. 

The following graphs are taken from Table 5 and show the results for the three major 
reading achievement measures used in the study. 



Figure 1 



TEST OF EARLY READING ABILITY 

MEAN NCE 




* No significant differences. 



Figur e 2 



RUNNING RECORD LEVEL MEAN 

SCORE 




* No significant differences. 
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Figure 3 




* No significant difference between Open Court and Phonics in Context. Significant difference 
between Open Court and Control and between Phonics in Context and Control. 






Table 5 

Reading Achievement Posttest Means 



Posttest Variables 
TERA-2 

Running Record 

SAT Total Reading 

SAT Reading Comprehension 

SAT Word Reading 

SAT Word Study Skills 



open Court 
Integrated 
Model(OC) 


Phonics in 
Context(PC) 


63.3 


60.7 


N=93 


N=94 


6.8 


6.6 


N=82 


N=89 


52.2 


49.0 


N=78 


N=87 


51.7 


47.6 


N=78 


N=87 


51.2 


49.8 


N=78 


N=87 


53.1 


50.5 


N=78 


N=87 



ControKC) 


Significant 

Group 

Differences 


59.3 

N=86 


None 


6.5 

N=82 


None 


42.3 

N=83 


(OC & C)** 
(PC & C)** 


43.7 

N=83 


(OC &C)** 


44.5 

N=83 


(OC & O* 


39.5 

N=83 


(OC &C)** 
(PC & C)** 



* Statistically significant at a 95% confidence level N=Number of students 

** Statistically significant at a 99% confidence level 

The highest observed mean TERA-2 posttest score was for the Open Court group followed 
closely by the Phonics in Context group (Figure 1). However, the difference among all 
three groups was not statistically significant. The 2.6 NCE difference between the mean 
NCE for Open Court and that of the Phonics in Context group is about one item out of a 
maximum possible score of forty-six. 

For the Ru nnin g Record measure of oral reading acliievement, the differences were not 
large enough to be statistically significantbetween any ofthe groups (Figure 2). The mean 
level for each of the three groups was at a normal exid of first grade level. 

Stanford Acliievement Test results (Figure 3) show that both the Open Court and 
Phonics in Context groups scored significantly higher than the control group but they did 
not differ significantly from each other. To provide some perspective, the difference 
between the mean score of 52.2 for Open Court and 49.0 for Phonics in Context translates 
to about three items on the seventy item Total Stanford Reading Test. 

Another way to look at performance of the groups is to compare their results on the 
Reading Comprehension test to districUvide results from the last time we gave the test in 
all first grade classrooms. The mean scores for Reading Comprehension were 51.7 for 
Open Court, 47.6 for Phonics in Context, and 43.7 for the Control group. The results for 
all three groups are either equal to or higher than the districtwide mean of 42 registered 
in the spring of 1993. 
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Teacher Interviews 

Four of the five Open Court (OC) teachers were present at the interview session on April 
24, 1995. Seven of the eight Phonics in Context (PC) teachers were present for their 
interview session on April 25. The following summary contains responses to the questions 
that were asked during the interview although the session was rmstructured enough that 
teachei’S were free to make any comments that they wanted to make. 



A summary of teacher responses is given below: 



How adequately did the training prepare you to implement the program? 

Both OC and PC groups were enthusiastic in their support of the training and the persons 
conducting the training sessions. Although a few sessions were missed in both groups, it 
did not appear to be substantial enough to be of major concern for the study. Because of 
the different nature of the two programs, the OC group felt generally confident that they 
received what they needed to implement the program while the PC group felt that ongoing 
inservice would have been desirable since they were learning new teaching methods and 
strategies. The PC group also believed that they did not have adequate funding to 
reproduce and purchase materials and felt that this was a drawback for their program. 

The PC group believed that the training would have best been conducted prior to the 
beginning of the school year and, therefore, the program wasn’t fully in effect for the entire 
school year. The training didn’t begin until October and major parts of the program were 
still being presented after mid-year. For OC, the major two-day presentation of the 
materials and how to use them was done in August so implementation came much closer 
to being in effect for the full year. 

How quickly have teachers gotten “up to speed” in implementing the program? 

For OC, teachers had the two major inservice days in August prior to school starting. Even 
though there were follow-up sessions in the fall and spi’ing, the major elements of the 
progi’am were introduced in time to begin implementation early in the year. Nonetheless, 
with teacher changes occurring early in the year, the trainer believed it wasn’t imtil 
November that students could benefit fully from the progr 2 un. Although most teachers 
reported adapting quite well to the program, at least one teacher reported having more 
difficulty. One teacher at a year-round school started a couple of months into the school 
year and, therefore, was behind the other teachers in the number of lessons delivered. 
For PC, the first training session wasn’t until September 30th with several sessions 
occurring after December. Teachers did not believe they had enough exposure to concepts 
and skills to expect much of an impact until the second semester of the 1994-95 school year. 
Although a couple of the teachers were implementing at a high level by the spring, a couple 
of others were still struggling. One reason given were the many and varied demands being 
placed on teachers for other staff development and assessment collection activities, 
couple of the PC teachers also experienced health related and other personal problems that 
caused them to miss more than one training session and delayed implementation. An 
illness in the family of one of the OC teachers may have affected implementation in her 
classroom as well. 
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What have been the strengths and weaknesses of the program as you see them? 
Open Court ; 



Strengi;hs: The teachers were very impressed with the trainer and her skills and would 
recommend her for any future inservices. OC teachers knew that they had systematically 
introduced everything to the students and while middle and high level readers don’t have 
to depend as much on phonics, it is important for the lower level readers. The students 
enjoyed the program and were very enthusiastic during the OC sessions. Parents also 
made many positive comments about the program. One teacher indicated that in her 
judgment, her students were doing better than her previous classes. 

All but one of the OC teachers had gone through the district’s literacy program and felt 
comfortable with the integration of OC into a whole language program. 

Another perceived strength is that two of the teachers had used OC in other districts before 
coming to RUSD. 

Weaknesses: The controlled vocabulary books were not really necessary and were not 
used very much although they may have been best for the lower level readers who were 
struggling. The books were seen as boring and the vocab’Jary as contrived so other 
literature was used for the most part. Also, the time allocated to OC may have been too 
much at about an hour per day. The group recommended that the amount )f time spent 
on the delivery of phonics be shortened to four days per week and less than an hour per day. 

In addition, one teacher commented that wliile she had hoped that the OC program would 
have especially helped the lowest achieving students, she observed that this group still 
stmggled even with the OC program. Another teacher emphasized that she didn’t see how 
she could attribute student performance to the OC program itself since so many things 
changed in her instructional program. Specifically mentioned was the fact that students 
were now being allowed to take books home for the first time and this could be expected 
to make a substantial difference. 

The running record levels were seen as confusing because of the color-coded levels. 
Teachers also noted that the Wright and Rigby books, used for the running records and the 
guided reading groups, were not readily available. 



Phonics in Context ; 

Strengths: Teachers felt that the day-long training was better than the part-day training. 
All felt that they had access to language arts experts for the training. One teacher 
mentioned that her Idds were handling the Houghton-Mifflin literature materials better 
than previous classes. Others mentioned that students were seeing parts of words and 
words-within-words better now. They also mentioned a sense of enthusiasm among 
students that the changes in their instruction had created. At Liberty, all first grade 
teachers went through the training even though not all were part of the study. It was felt 
that this was a special strength at Liberty since all teachers at the grade level can be “on 
the same page” so to speak and could support one another. Another perceived strength of 
the program was that it was very much aligned with the Language Arts Framework and 
a whole language philosophy. 




Weaknesses: Teachers had to prepare and reproduce large volumes of materials which 
they felt should have been provided by the district. The running record levels were seen 
as confusing because of the color-coded levels. While the running record was important for 
the program, they didn’t think they did it especially well given the time requirement. This 
was the perception even though all participating teachers v/ere provided two days with a 
substitute at the end of both the first and third trimesters to complete the running records. 
Another perceived weakness was that the Wright and Rigby books were not available in 
some classrooms until December so students could not take books home mitil then. 

In summary, some of the major conclusions from the Teacher Interviews included: 

• Both Open Court and Phonics in Context teachers were enthusiastic about the 
quality of training. 

• Both Open Court and Phonics in Context teachers believed their teaching had 
improved and their students were more successful as a result of the respective 
programs. 

• Although the Open Court group felt that they didn’t get things running smoothly 
until a couple of months into the school year, the Phonics in Context group indicated 
that major components of their program were still being delivered at mid-year. As 
a consequence, the PC group believed that results of the study would reflect only a 
partial implementation. 

• Both groups indicated that increased exposure of students to reading at home as a 
result of a new district emphasis could account for part of the student progress. 

• The Open Court group while adamant, in general, about continuing the use of the 
materials next year, believed that less time should be spent on the Open Court 
component. 

• Both the Open Court and Phonics in Context groups each included one year-round 
school which was in session for two to three months prior to the inception of the staff 
development inservices. This delayedimplementationcoiildbe expected to negatively 
impact any gains made as a result of the respective programs at these schools. 
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Teacher Survey 



Teachers participating in the field test were surveyed during the first trimester in order 
to construct a profile of the teachers in the three groups. The survey covered: (1) 
backgroimd and teaching experience including recent staff development, (2) the level of 
support from aides and volunteers and how much time is typically devoted to reading 
instruction, and (3) extent to which students spend time in other settings through teaming 
and special programs like Chapter 1. Results of the survey are given below: 



Years teaching at school: 


Average 


Open Court 


11 


Phonics in Context 


15 


Control 


9 



Recent Grade Levels taught: 



All teachers had taught at the primary level for the past three years and all but four 
teachers surveyedhadtaughtfirstgrade for atleasttwoofthose three years. Consequently, 
there appears to be little difference between the groups on this factor. 



How were you assigned to the class?: 

All but a couple of teachers indicated that they usually taught at first grade and that is 
why they were assigned to the class. Two teachers indicated that they did not volunteer 
but were assigned to the class by their principal (Phonics in Context). There appears to 
be little difference between the groups on this factor. 



Do you have instructional aides in your classroom?: 

None of the Open Court or Phonics in Context teachers indicated that they had aides in 
their classrooms. Three of the six control group teachers reported that they had aides 
(two had 1 and one had 2). These aides spent 30 minutes a day five days a week in two 
of the three classes and the other had a 3-hour aide five days a week. 



How many volunteers do you typically have?: 

All teachers reported at least one volunteer and typically it was two or three. A couple 
of the teachers reported five to eight volunteers (one Phonics in Context; three control, 
and one Open Court) The average amount of hours per day and days per week were: 

Days Per Week Hours Per Day 
Open Court 1 2 

Phonics in Context 1 2 

Controls 4 2 

There appears to be a substantial difference between the Control group and the others 
in amount of volunteer time with the Control group classes averaging four days per week 
having volunteers while the two treatment groups reported approximately 1 day per 
week. _ 

13 

n; 



Number of minutes you read to your class: 

Here there were also some differences reported as follows: 



Open Court 
Phonics in Context 
Controls 



Days Per Week 
5 
5 
5 



Minutes Per Day 
21 
33 
30 



While there was little difference between the Control and the Phonics in Context groups, 
the Open Court teachers reported spending less time reading to the class. It should be 
noted that these responses were giy en early in the school year and should not be assumed 
to reflect what was happening later in the year when the respectiye programs were fully 
implemented. 



Do you team or exchange students with another teacher for Language Arts?: 

Only one of the Open Court teachers and one of the Phonics in Context teachers reported 
that they team teach for Language Arts. None of the Control group teachers reported 
team teaching for Language Arts. 



Do any of your students go to a Chapter 1 or RSP teacher for pull-out 
programs?: 

One Open Court (7 students), four Phonics in Context teachers (an ayerage of 7 
students), and one Control group teacher (2 students) indicated that they had students 
who were pulled out of class for Chapter 1 or RSP programs. 



Recent Staff Development: 

All fiy e of the Open Court teachers reported haying gone through the district’s Language 
Arts Literacy training and two indicated haying had the Wright Group training. Other 
inservice indicated included: Improying first grade methods, reading methods, TESA, 
McCracken, reading specialist, and a masters degree at UCR. 

All six of the Phonics in Context teachers indicated haying had the Literacy Training ^d 
three reported training in the running record. Three reported Wright Group training 
and all six reported McCracken training. 

Of the Control group teachers, fiye reported having the Literacy Training and two 
reported McCracken and Wright Group training. Other inservices indicated were: 
running record, reading recoyery, whole language, and guided reading. 

Clearly, there was no difference between any of the groups in terms of exposure to the 
first-year Literacy training. All three groups appear to haye similar staff deyelopment 
backgrounds as evidenced by some of the common inservices attended recently. 
Consequently, there appears to be little difference between the groups on this factor. 







Parent Telephone Survey 

During April and May, a telephone survey was conducted wherein an attempt was made 
to contact all parents of students who were enrolled in the field test classrooms since the 
beginning ofthe school year. The survey consisted of six questions. Questions 1 through 
d asked about the child’s reading behavior at home while questions 4 and 5 asked about 
the child s attitude toward reading as perceived by the parent. Question 6 asked about 
general satisfaction with their child’s reading progress. Parents were also given the 
chaince to answer an open-ended question regarding the main reason for their satisfaction 
or dissatisfaction as a follow-up to question 6. 

Of the 465 students who took the TERA-2 pretest, 362 parents, were successfully 
contacted. However, this represented a high percentage ofthe students remaining in the 
field test classrooms at the end of the year as shown in Table 6 which compares the 



TABLE 6 

Parent Response Rate 




Parents Contacted 


TERA-2 IMav) 


Open Court 


114 


126 


Phonics in Context 


147 


142 


Control 


101 


99 



The number and percent of parents responding to each response option for the survey 
questions are presented below. The average score (mean) on the four or five point scale 
used in the survey is also reported on the right-hand side of each table. This enables one 
to evaluate the statistical significance ofthe differences in average ratings between the 
groups. 



Results are as follows: 



1. How many days per week on average does your child either read or get read 
to at home by an adult or other family member for homework, before bedtime, 
or at other times? 



Open Court 
Ph. In Context 
Control 



Evervdav. 


5-6 davs. 


3-4 davs 


2 davs 


1 dav 


Average 


43(38%) 


41(36%) 


24(21%) 


5(4%) 


1(1%) 


4.05 


42(29%) 


60(41%) 


36(25%) 


9(6%) 




3.92 


30(29%) 


32(32%) 


32(32%) 


7(7%) 




3.84 



*No significant differences among the group averages. 



15 



O 

lERlCi 






i Vi - 











2. How many minutes on average does your child either read or get read to 
on these days? 

30+ min . 20-25 min . 15 min. 10 min. 5 min. Averag e 



Open Court 34(30%) 43(38%) 31(27%) 

Ph. In Context 39(27%) 52(36%) 41(28%) 

Control 16(16%) 29(29%) 44(44%) 



6(5%) 3.92 

13(9%) 1(1%) 3.79 

10(10%) 2(2%) 3.47 



* Significant differences between the averages for OC and C and between PC and C. 
No significant difference between OC and PC. 



3. How many days per week on the average does your child pick-up a book 
on his/her own to read or ask someone to read it to him or her? 


Evervdav 


5-6 days 


3-4 days 


2 davs 


1 dav Aveiage 


Open Court 36(32%) 
Ph. In Context 54(37%) 
Control 33(33%) 


23(20%) 

36(25%) 

15(15%) 


40(35%) 

33(22%) 

24(24%) 


5(4%) 

16(11%) 

18(18%) 


10(9%) 3.61 

8(5%) 3.76 

11(11%) 3.41 


* No significant differences among the 


group averages. 







4. How much docs your child like to be read to by 


an adult or other family 


member? 


Very Much 


Some 


Not much 


Very little 


Average 


Open Court 


95(84%) 


16(14%) 


1(1%) 


1(1%) 


3.81 


Ph. In Context 


102(69%) 


33(22%) 


9(6%) 


3(2%) 


3.59 


Control 


75(75%) 


21(21%) 


2(2%) 


2(2%) 


3.69 


* Significant differences between the averages for OC and PC. 






5. i?ow much does your child like to read on his/her own? 






Very Much 


Some, 


Not much 


Verv little 


Average 


Open Court 


71(63%) 


36(32%) 


5(4%) 


1(1%) 


3.57 


Ph. In Context 


81(56%) 


36(25%) 


11(8%) 


16(11%) 


3.26 


Control 


56(56%) 


33(33%) 


4(4%) 


7(7%) 


3.38 



* Significant differences betw'een the averages for OC and PC. 




6. In general, how satisfied have you been with your child’s progress in 
learning to read at school this year? 





Very 

Satisfied 


Satisfied 


Dissatisfied 


Very 

Dissatisfied 


Average 


Open Court 


72(63%) 


38(33%) 


3(3%) 


1(1%) 


3.59 


Plx. In Context 


78(54%) 


52(36%) 


10(7%) 


4(3%) 


3.42 


Control 


58(58%) 


31(31%) 


6(6%) 


5(5%) 


3.42 



* No significant differences among the gro up averages. 

These results show a somewhat higher degree of parent satisfaction reported by the 
Open Court parents, but the differences were not statistically significant. Student 
enjoyment of reading is also somewhat higher for Open Court as measured by questions 
4 and 5 but higher for Phonics in Context as measured by question 3. Questions 1 and 
2 suggest that students may have read more at home in the Open Court group. 

Parent Comments 

Even though not all parents commented on why they were satisfied or dissatisfied with 
their child’s reading progress in 1994-95, the vast majority did state a reason. The 
percentage of parents making positive comments mirrored the results of the multiple 
response question (#6) with 96%, 88%, and 87% making positive comments for the Open 
Court, Phonics in Context, and the Control group, respectively. 

For Open Court, positive comments fell into five major categories with the percentage 
of total positive comments falling into each category as follows: 

Child reading well/rnaking good progress — 43% 

Child has good teacher — 7% 

Child hkes/is interested in reading — 22% 

Specifically likes Open Court (phonics program) — 18% 

Likes how child brings books home every night — 10% 

Negative comments were: 

• Child is advanced but will not let him move on — ^keeps all at same level 

• Child needs more one-on-one teaching 

• Materials too easy and require no imagination 

• Child hasn’t learned how to read yet — difficult to understand how reading program 

was presented 

For Phonics in Context , positive comments were as follows: 

Child reading well/making good progress — 62% 

Child has good teacher — 15% 

Child likes/is interested in reading — 19% 

Negative comments included: 



• Not getting enough enrichment from GATE 

• Child can’t read well enough (7) 

• Teacher needs to help more (2) 

• Not enough phonics (1) 

• Difference between what teacher says and v/hat I see at home 

• Child can’t stay still so he can’t read 

• Doesn’t bring reading material home 

• I was responsible for child’s reading — ^not school 

For the Control group, positive comments were as follows; 

Child reading well/making good progress — 68% 

Child has good teacher — 16% 

Child likes/is interested in reading — 16% 

Negative comments included; 

• Does not teach the right way 

• Child not getting enough attention 

• School doesn’t emphasize reading enough (2) 

• Child can’t read w’ell enough (2) 

• School says child has handicap in reading 

• Class needs to be taught all in one level in reading 

• Because they are not helping improve his reading 

• Doesn’t feel that phonics methods are being used; doesn’t like pictures used in reading 

Limitations of the Study 

It should be acknowledged that in any study conducted in a natural setting like school 
classrooms, there exist several factors that cannot be adequately controlled and that 
could lead to alternative interpretations of the results. 

Although pretest achievement levels and parent socio-economic status w^ere controlled 
statistically, there are potentially many other things that could have affected the 
outcomes. Certainly one of the most problematic is the difficulty in adequately controlling 
for differences in school climates and teacher talent. Although the profile developed for 
teachers in the study showed several similarities and did not give a gross indication of 
teacher differences, the existence of such differences cannot be ruled out. Also the 
teachers in the two treatment groups were, for the most part, self-selected and especially 
enthusiastic about participating. Thus they may not be representative of first-grade 
teachers districtwide. 

In addition, since many components of the literature-based program implemented by the 
Phonics in Context group were also present in the Open Court Program, it is very difficulc 
to separate out the effects of the phonics component per se on student achievement. 
Likewise, the effects of participation in Chapter 1 programs could be expected to affect 
reading outcomes, especially for the Phonics in Context group which included 20 Chapter 
1 students, as opposed to three Chapter 1 students in the Open Court group. The control 
group did not include Chapter 1 students. 
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Conclusions: 



The observed mean scores on the posttest TERA-2 and the Running Record levels were 
highest for Open Court followed closely by Phonics in Context. However, these differences 
were not large enough to project confidently to a larger population of first graders. 
Performance on the Stanford Achievement test showed the same pattern with no significant 
differences between the two treatment groups. Differences between Open Court and the 
Control group were, however, large enough to be stp "tically significant. Differences 
between the Phonics in Context and Control groupr .re significant for Stanford Word 
Study Skills and Stanford Total Reading. 

Teacher interviews revealed staff who were equally enthusiastic about their respective 
programs. Both groups had members indicating that they felt students had made more 
progress than past classes. Key points noted also included the fact that the Phonics in 
Context group believed that test results would only reflect a partial year implementation 
given the fact that key parts of the inservice did not occur imtil mid-year. In addition, since 
the program is really attempting to change the way teachers approach instruction, the 
skills taught take time to hone and perfect. While the Open Court group was enthusiastic 
about continuing use of the materials, it was thought that the hour per day required for the 
phonics component was somewhat excessive and should be reduced. 

Parents surveyed were generally satisfied with their child's reading progress with the 
Open Court parents somewhat more positive. These differences regarding satisfaction did 
not prove, however, to be statistic^ly significant. Questions about how much their 
children liked to read were responded to more positively by the Open Court parents and 
this result was statistically significant. 

Questions about the backgroimd of the teachers participating in the study and classroom 
characteristics apart from the field test programs showed differences regarding the 
number of volunteer hours utilized (the Control group had more volimteers) and the 
number of minutes spent reading to the students (the Open Court group indicated less 
minutes early in the year). However, other questions relating to recent staff development 
and experience teaching at the first grade revealed little difference between the teacher 
groups. 

In summary, this study clearly shows that at least on the measures of reading achievement 
used in this study, students appear to be responding well to both the Open Court 
integration with whole language as well as to the Phonics in Context methods. In addition, 
each treatment group outperformed the Control group students. 




Descriptive statistics for the three field test groups showing various demographic and 
other characteristics of the samples are presented below. 



TABLE 2 

Selected Demographics 




Variable 


Open Court 


Phonics in Context 


ContrjQl 


# of RSP Students 


0 


1 


0 


# of LEP Students 


4 


2 


5 


# of Chapter 1 Students 


3 


20 


0 


# of Reading Recovery Students 


7 


12 


5 



TABLE 3 

Correlations between Variables 
and Reading Achievement 


Posttest Reading Measures 


TERA-2 (Pretest) 


SES 


TERA-2 


.99 


.30 


Running Record 


.92 


.58 


Stanford Total Reading 


.87 


.64 



TABLE 4 

Average Scores on TERA-2 Pretest and SES 










Significant 






Phonics in 




Group 


Variable 


Open Court 


Context 


Control 


Differences 


TERA2 (Pretest) 


48.7 


48.7 


51.8 


None 




N=94 


N=94 


N=86 




Parent 










Occupational 


3.8 


3.9 


3.8 


None 


Level (SES) 


N=94 


N=94 


N=86 





